Migrate OTel observer LLM span to typed event#142
Merged
Conversation
Drive the openarmature.llm.complete span lifecycle from the typed LlmCompletionEvent on the success path. The span opens and closes in one shot at typed-event arrival, with start_time back-dated by latency_ms so the duration reflects the adapter-boundary measurement rather than dispatcher queue delay. Failure-path spans continue to fire from the sentinel NodeEvent pair (the typed event is success-only per proposal 0049 alternative 3). The §5.5 attribute set is unchanged. Flip OpenAIProvider.populate_caller_metadata default from False to True so the bundled OTel and Langfuse observers can emit the §5.6 openarmature.user.<key> span-attribute family without callers having to opt in. The spec-defined opt-in mechanism is preserved; only the python default flips.
There was a problem hiding this comment.
Pull request overview
Migrates the OpenTelemetry observer’s successful LLM span lifecycle (openarmature.llm.complete) from the legacy sentinel NodeEvent pair to the typed LlmCompletionEvent, including back-dating the span start time from latency_ms to reflect adapter-boundary latency. Also flips OpenAIProvider.populate_caller_metadata default to True so caller invocation metadata flows into observers by default.
Changes:
- OTel observer now opens+closes the success-path LLM span on typed
LlmCompletionEventarrival and back-datesstart_timeusinglatency_ms; sentinel events remain for the error path during the dual-emit window. - OpenAI provider default now populates
caller_invocation_metadata(opt-out remains available). - Updates/adds unit tests and introduces a shared typed-event test helper.
Reviewed changes
Copilot reviewed 7 out of 8 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
src/openarmature/observability/otel/observer.py |
Drive success-path LLM spans from LlmCompletionEvent (with latency back-dating) and keep sentinel-based error spans. |
src/openarmature/llm/providers/openai.py |
Flip populate_caller_metadata default to True and update rationale comments. |
src/openarmature/graph/events.py |
Update LlmCompletionEvent documentation to reflect Python default caller-metadata behavior. |
tests/unit/test_observability_otel.py |
Update observer tests to use typed events; add duration/latency and gating tests. |
tests/unit/test_llm_provider.py |
Update tests for the new default caller-invocation-metadata behavior and opt-out. |
tests/_helpers/typed_event.py |
Add helper to construct LlmCompletionEvent instances for tests. |
CHANGELOG.md |
Document typed-event OTel span lifecycle change and OpenAI default flip. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Pass the typed LlmCompletionEvent to attribute_enrichers on the success path instead of None; widens the enricher signature to accept LlmCompletionEvent so enrichers can read LLM-call context that the sentinel-pair path used to expose via the closing NodeEvent. Switch the shared typed-event test helper's default node_name and namespace to "ask" so they reflect the calling-node semantics the typed event documents, not the legacy sentinel value.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
openarmature.llm.completespan lifecycle from the typedLlmCompletionEventon the success path. Span opens + closes in one shot at typed-event arrival, withstart_timeback-dated bylatency_msso the duration reflects the adapter-boundary measurement rather than dispatcher queue delay.NodeEventpair (the typed event is success-only per proposal 0049 alternative 3). The §5.5 attribute set is unchanged.OpenAIProvider.populate_caller_metadatadefault fromFalsetoTrueso the bundled OTel and Langfuse observers can emit the §5.6openarmature.user.<key>span-attribute family without callers having to opt in. The spec-defined opt-in mechanism is preserved; only the python default flips.Scope
PR 3b of the proposal 0049 + 0057 rollout (PR 3a landed in #141). Follow-up:
The provider still emits both the sentinel
NodeEventpair AND the typedLlmCompletionEventduring v0.13.0 per the dual-emit window. The sentinel pair drops in v0.15.0.Behavior change worth calling out
OpenAIProvider(populate_caller_metadata=...)default flip is a public-API behavior change. Callers passing the kwarg explicitly are unaffected; callers relying on the default now getcaller_invocation_metadatapopulated on every typed event, which surfaces asopenarmature.user.*span attributes on the OTel LLM span. Passpopulate_caller_metadata=Falseto suppress the snapshot if no downstream consumer needs it.Test plan
uv run pytest tests/unit/test_observability_otel.py— 50 passed (added 3 new tests: span duration accuracy,latency_ms=Nonezero-duration fallback,disable_llm_spanson the typed path).uv run pytest tests/— 1203 passed across the full suite.uv run pyrightclean.uv run ruff check+ruff format --checkclean.026-otel-caller-supplied-metadatapasses (caller-metadata default flip restores the OTel span attribution that the migration would have otherwise lost).